{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Distributed Benign Training\n",
    "\n",
    "`advsecurenet` library provides a simple way to train a benign model both in a distributed and non-distributed way. In this notebook, we will train a benign model in a distributed way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a Configuration File\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Create a Configuration File\n",
    "\n",
    "As a first step, you need to have a default configuration file. You can create a configuration file by running the following command:\n",
    "    \n",
    "```sh\n",
    "advsecurenet utils configs get -c train_config.yml -s -o ./default_train_config.yml\n",
    "```\n",
    "\n",
    "Here, we are also providing the `-o` flag to specify the output directory where the configuration file will be saved. You can also specify the output directory by passing the `-o` flag followed by the path to the directory where you want to save the configuration file. If you don't provide the `-o` flag, the configuration file will be saved in the current working directory with the name `train_attack_config.yml`. If the file already exists, it will be overwritten. If the output path is a directory, the configuration file will be saved in that directory with the name `train_attack_config.yml`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving default config to ./default_train_config.yml\n",
      "Generated default configuration file train_config.yml!\n"
     ]
    }
   ],
   "source": [
    "!advsecurenet utils configs get -c train_config.yml -s -o ./default_train_config.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1m**************************************************\u001b[0m\n",
      "\u001b[31m\u001b[1mDefault configuration file for the train_config.yml:\n",
      "\u001b[0m\n",
      "\u001b[32m{\n",
      "    \"model\": {\n",
      "        \"model_name\": \"resnet18\",\n",
      "        \"num_input_channels\": 3,\n",
      "        \"num_classes\": 10,\n",
      "        \"pretrained\": true,\n",
      "        \"weights\": \"IMAGENET1K_V1\",\n",
      "        \"is_external\": false,\n",
      "        \"random_seed\": null,\n",
      "        \"path_configs\": {\n",
      "            \"model_arch_path\": null,\n",
      "            \"model_weights_path\": null\n",
      "        },\n",
      "        \"norm_config\": {\n",
      "            \"add_norm_layer\": false,\n",
      "            \"norm_mean\": [\n",
      "                0.5,\n",
      "                0.5,\n",
      "                0.5\n",
      "            ],\n",
      "            \"norm_std\": [\n",
      "                0.5,\n",
      "                0.5,\n",
      "                0.5\n",
      "            ]\n",
      "        }\n",
      "    },\n",
      "    \"dataset\": {\n",
      "        \"dataset_name\": \"cifar10\",\n",
      "        \"num_classes\": 10,\n",
      "        \"train_dataset_path\": null,\n",
      "        \"test_dataset_path\": null,\n",
      "        \"preprocessing\": {\n",
      "            \"steps\": [\n",
      "                {\n",
      "                    \"name\": \"ToImage\",\n",
      "                    \"params\": null\n",
      "                },\n",
      "                {\n",
      "                    \"name\": \"ToDtype\",\n",
      "                    \"params\": {\n",
      "                        \"dtype\": \"torch.float32\",\n",
      "                        \"scale\": true\n",
      "                    }\n",
      "                }\n",
      "            ]\n",
      "        },\n",
      "        \"download\": true\n",
      "    },\n",
      "    \"dataloader\": {\n",
      "        \"train\": {\n",
      "            \"num_workers\": 4,\n",
      "            \"shuffle\": true,\n",
      "            \"drop_last\": false,\n",
      "            \"pin_memory\": true,\n",
      "            \"batch_size\": 32\n",
      "        },\n",
      "        \"test\": {\n",
      "            \"num_workers\": 4,\n",
      "            \"shuffle\": true,\n",
      "            \"drop_last\": false,\n",
      "            \"pin_memory\": true,\n",
      "            \"batch_size\": 32\n",
      "        }\n",
      "    },\n",
      "    \"device\": {\n",
      "        \"use_ddp\": false,\n",
      "        \"processor\": \"cpu\",\n",
      "        \"gpu_ids\": null\n",
      "    },\n",
      "    \"training\": {\n",
      "        \"epochs\": 10,\n",
      "        \"learning_rate\": 0.001,\n",
      "        \"optimizer\": \"adam\",\n",
      "        \"optimizer_kwargs\": {\n",
      "            \"weight_decay\": 0.0001\n",
      "        },\n",
      "        \"scheduler\": \"LINEAR_LR\",\n",
      "        \"scheduler_kwargs\": null,\n",
      "        \"criterion\": \"cross_entropy\",\n",
      "        \"save_final_model\": true,\n",
      "        \"save_model_path\": null,\n",
      "        \"save_model_name\": null,\n",
      "        \"save_checkpoint\": false,\n",
      "        \"save_checkpoint_path\": null,\n",
      "        \"save_checkpoint_name\": null,\n",
      "        \"checkpoint_interval\": 1,\n",
      "        \"load_checkpoint\": false,\n",
      "        \"load_checkpoint_path\": null,\n",
      "        \"verbose\": true\n",
      "    }\n",
      "}\u001b[0m\n",
      "\u001b[34m\u001b[1m**************************************************\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Let's check the content of the default config file\n",
    "!advsecurenet utils configs get -c train_config.yml -p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Modify the Configuration File\n",
    "\n",
    "The default configuration file is capable of running the training on the `CIFAR10` dataset using the `ResNet18` model.\n",
    "\n",
    "You can modify the configuration file to run the training on a different dataset, model, or change other hyperparameters. You can also add new hyperparameters to the configuration file.\n",
    "\n",
    "For this example, we will modify the configuration file to run the training on the `Cifar100` dataset using the `Vgg16` model. We will also change the number of epochs to 1, the learning rate to 0.01, and the batch size to 64. Finally, we will set the `use_ddp` flag to `True` to run the training in a distributed way.\n",
    "\n",
    "\n",
    "\n",
    "You can find the updated configuration file [here](train_config.yml).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Train the Model\n",
    "\n",
    "Now, you can train the model by running the following command:\n",
    "\n",
    "```sh\n",
    "advsecurenet train -c train_config.yml\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!advsecurenet train -c train_config.yml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations! You have successfully trained a benign model in a distributed way using the `advsecurenet` library. You can now use this model for inference or further training."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "adv2",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
